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DELTA-EXACT  LOWER  CONFIDENCE  BOUNDS 
FOR  SERIES  SYSTEM  RELIABILITY 


I.  Introduction 

This  report  contains  some  recent  modifications  to  the  Likelihood  Ratio 
Method  when  computing  lower  confidence  bounds  for  system  reliability  from 
binomial  component  data.  It  is  shown  that  these  modifications  produce  bounds 
which  appear  to  have  those  properties  necessary  for  accurate  application  to 
binomial  data  usually  found  in  practice,  that  is  from  structures  of  order 
greater  than  three  binomial  component  data  with  unequal  sample  sizes  and  a 
paucity  of  failures. 

Journal  articles  on  comparisons  of  various  confidence  bound  techniques 
for  series  system  reliability  from  binomial  component  data  invariably  only 
compare  specific  component  sample  size  (usually  equal)  and  specific  success 
data  for  systems  of  order  two  and  three.  (For  example  see  Mann  and  Grubbs 
[8],  Myhre  and  Saunders  [11]  and  Winterbottom  [12]).  These  comparisons  are 
not  adequate  for  data  from  the  majority  of  systems  seen  in  practice.  Therefore, 
for  this  report  we  compare  various  confidence  bound  methods  by  means  of 
simulation.  That  is,  assume  £  =  (p^»  •••  P^)  and  H  “  (n^,  •••  are  given 
and  simulate,  say  500  times  data  of  the  form  (x^,n^),...,  •  From 

these  500  sets  of  data  calculate  500  lower  (l-a)%  confidence  bounds  for  each 
method  that  is  being  compared.  Comparisons  of  the  various  methods  are  made 
using  the  empirical  confidence  bound  distributions  obtained  from  the  observed 
lower  confidence  bounds. 


(1)  A  discussion  of  the  number  of  simulations  performed  and  why  is  given 
on  page  10. 
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Simulation  results  and  the  concept  of  6-exact  bounds  demonstrate  why 
the  Modified  Likelihood  Ratio  Methods  yield  bounds  which  are  preferable  to 
those  obtained  using  methods  such  as  Approximately  Optimum,  Modified 
Maximum  Likelihood  and  Modified  Log  Gamma. 

II .  Desirable  Properties  of  Lower  Confidence  Bounds  on  System  Reliability 

Many  expensive  systems  such  as  missiles,  are  produced  to  complete  one 
task  only  and  they  must  be  certified  to  the  purchaser  to  be  highly  reliable 
even  though  they  often  may  not  be  tested  directly.  In  such  a  case  the 
producer  will  test  the  individual  components  or  subsystems  of  the  system  and 
obtain  a  point  estimate  of  systems  reliability,  R,  from  the  resulting  binomial 
data:  x^  successes  out  of  n^  trials  for  1  <  i  <  k,  where  k  is  the  number  of 
components.  Further,  most  contracts  require  an  interval  estimate  in  the  form 
of  a  lower  confidence  bound  on  system  reliability.  The  estimation  problem 
of  obtaining  a  point  estimate  for  system  reliability  is  much  easier  than  that 
of  obtaining  an  interval  estimate.  Even  though  the  literature  contains  a 
number  of  methods  for  computing  interval  estimates  for  the  reliability  of 
series  systems,  these  methods  usually  do  not  produce  acceptable  bounds  for 
highly  reliable  systems  where  the  data  are  disparate  component  sample  sizes 
and  a  paucity  of  component  failures. 

There  are  a  number  of  properties  that  have  been  used  historically  in 
the  mathematical  literature  as  desirable  properties  for  lower  confidence 
bounds: 

A  confidence  interval  R  (x^,...,x^)  is  said  to  be  unbiased  if  the 
probability  of  the  interval  covering  the  true  parameter  value  R  is  greater 
than  or  equal  to  the  probability  of  its  covering  any  false  value.  That  is, 
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for  confidence  level  1-a,  P  [R  (x^,...,^)  <  R']  <  1-a  for  all  R'  <  R 
and  for  all  nuisance  parameters  p  ,...,p  . 

-L  K 

A  uniformly  most  accurate  (UMA)  confidence  interval,  in  a  specified 
class  of  intervals,  is  based  on  the  acceptance  region  of  a  hypothesis  test 
that  is  uniformly  most  powerful  in  the  corresponding  class  of  tests. 

A  confidence  interval  is  optimum  if  it  is  a  UMA  unbiased  confidence 
interval. 

A  lower  confidence  bound  of  level  1-a  for  R  is  exact  if  the  lower  limits 
obtained  in  random  sampling  are  less  than  or  equal  to  R  on  at  a 

proportion  1-a  of  occasions  in  the  long  run. 

A  lower  confidence  bound  for  R  is  consistent  if  for  every  e  >  0  and  all 
1  S  i  <  k 

lim  P  [  |  It  (x  ,  ...,x.)  —  R  J  <  e  ]  =  0 


There  are  also  a  number  of  properties  which  are  desirable  or  in  fact  even 
necessary  for  a  practical  application  of  confidence  bounds: 

The  producer  needs  a  confidence  bound  method  that  produces  bounds  that 
are  as  close  to  the  true  reliability  as  possible:  The  variance  of  the  confidence 
bounds  about  the  true  R  should  be  a  minimum. 

It  is  also  necessary  that  the  method  be  flexible  so  it  is  applicable  to 
the  variety  of  cases  that  arise  in  practice.  The  constraints  of  time  and 
expense  often  result  in  data  about  the  system's  components  that  consist  of 
a  variety  of  sample  sizes,  n^,  1  5  i  S  k.  The  sample  sizes  of  all  or  many 
componenLS  may  be  constrained  by  expense  or  configuration  to  be  small,  either 
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absolutely  or  with  respect  to  other  component's  sample  sizes.  Thus  a  method 
should  work  equally  well  for  data  due  to  equal  sample  sizes  or  to  a  variety 
of  sample  sizes  on  different  components.  It  is  also  necessary  to  have  a 
method  that  is  applicable  when  some  or  all  of  the  components  experience  no 
failures  in  testing. 

It  would  also  be  a  desirable  feature  if  the  method  were  easy  to  calculate; 
however,  for  most  cases  the  above  considerations  are  far  more  important. 

For  expensive,  highly  reliable  systems  it  seems  fair  to  assume  that  the  time 
and  expense  of  programming  and  running  a  more  desirable  method  will  be 
worthwhile.  Finally,  it  is  often  necessary  for  a  method  to  be  applicable  to 
more  complex  systems  as  well  as  for  the  simple  cases  of  series  or  parallel 
systems.  Of  course  if  one  method  could  be  shown  to  be  superior  in  a  special 
case  such  as  series  systems  but  not  applicable  to  other  cases,  it  would  be 
certain  to  be  used  in  that  special  case. 


III.  6-exact  Lower  Confidence  Bounds 

There  are  presently  a  variety  of  methods  used  to  compute  lower  confidence 
bounds  on  system  reliability  but  no  method  has  been  shown  to  be  universally 
best.  Some  methods  will  only  work  for  series  systems;  others  will  work  for 
all  types  of  systems  but  have  trouble  with  samples  containing  no  failures 
for  one  or  more  components.  Some  are  too  conservative  when  applied  to  highly 
reliable  systems  while  other  are  not  bounds  for  systems  of  moderate  reliability. 

In  order  to  enlarge  the  field  of  available  methods,  in  particular  to  admit 
tw>-  refinements  of  a  known  method  to  consideration  we  define  the  new  concept  of 
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6-exact  bound:  A  lower  confidence  bound  is  a  6-exact  (l-a)%  lower  confidence 
bound  if  in  the  long  run  (l-a)%  of  the  bounds  are  less  than  the  true  reliability 
plus  6.  For  example,  assume  the  true  reliability  of  a  system  is  .92,  6=.01 
and  80%  l.c.b.'s  are  computed  then  at  least  80%  of  the  interval  estimates 
would  be  less  than  .93.  If  such  a  method  were  used  on  a  particular  set  of 
data  and  a  6=.02,  90%  l.c.b.  were  estimated  to  be  .88  then  we  could  say  with 
90%  certainty  that  the  true  reliability  is  .86  or  higher. 

In  our  simulations  there  are  examples  in  which  a  method  that  gives  a 
6=.01,  90%  bound  can  be  much  better  than  a  method  that  is  a  6=.00,  90%  bound; 
for  the  .01,  90%  bound  may  have  standard  deviation  about  the  true  reliability 
of  .07  while  the  standard  deviation  of  the  6=.00,  90%  bound  is  .14.  Thus  the 
added  flexibility  of  being  a  6-exact  bound  may  pay  off  in  a  method  that  is 
much  closer  to  the  true  reliability. 


IV.  Methods  Used  for  Series  Systems  Comparisons 

There  has  been  much  recent  interest  in  developing  methods  that  compute 

lower  confidence  bounds  for  the  general  case  of  series  system  reliability. 

An  early  method  for  series  and  parallel  systems  that  has  found  acceptance 

was  presented  by  Madansky  in  1965  [7].  This  method  is  based  on  the  result, 

2 

due  to  Wilks,  that  -21n  L(P  )  is  distributed  asymptotically  as  Y  with  one 

o 

degree  of  freedom.  In  1968  the  method  was  extended  to  all  coherent  systems 
by  Myhre  and  Saunders  [10].  While  this  method,  as  documented  in  1968,  is  not 
exact  for  highly  reliable  systems  where  the  component  sample  size  is  small, 
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it  is  applicable  to  any  coherent  system  and  can  in  fact  be  shown  to  work 

m  k  a.  . 

for  systems  where  H(p)  =  Z  w.  (IT  P.  ^). 

j=l  J  i=l  1 

In  1972  Easterling  [2]  introduced  a  modification  to  the  maximum  likelihood 
method  (MMLI),a  method  which  is  based  on  the  asymptotic  normality  of  the  maximum 
likelihood  estimations.  This  method  was  shown  to  be  good  in  some  simple  cases 
but  that  the  MMLI  as  well  as  the  likelihood  ratio  method  have  problems  with 
data  that  might  arise  for  a  highly  reliable  system:  data  for  which  some 
components  have  no  failures. 

In  1974  Mann  and  Grubb  presented  the  Approximately  Optimum  method  (AO) 

2  3 

which  is  based  on  the  fact  that  P  [  R  i  exp{-m[l-(v/9m  )  +  za(v/3m)]  }]  =  l*-a 
where  m  and  v  are  the  mean  and  variance  of  the  posterior  distribution  of 
-In  R  and  -In  R  is  approximated  by  a  non-central  chi-square  distribution 
[  8  ].  This  method  is  restricted  to  series  (or  parallel)  systems  but  has  seen 
extensive  use  owing  to  the  fact  that  it  has  been  shown  to  be  close  to  exact. 

In  1968  Bor sting  and  Woods  [1]  introduced  a  method  based  on  fitting 

A 

-log  R  with  a  gamma  distribution.  Initially  it  was  used  only  for  series 
systems  but  it  has  been  modified  by  Woods  for  application  to  some  non-series 
systems.  Recently  Tomsky  further  modified  the  method  in  an  attempt  to  make 
it  less  conservative.  However,  in  testing  this  method  it  was  found  that  for 
a  given  set  of  sample  sizes  the  method  often  gives  a  lower  estimate  from  data 
with  no  failures  than  from  data  with  one  or  two  failures.  To  correct  this 
inconsistency  the  method  (MLGB)  was  modified  to  estimate  the  bound  for  the 
zero  failure  case  (x^n^  ,  1  £  i  <  k)  as  .01  larger  than  the  estimate  for 

th  case  in  which  exactly  one  failure  occurs  on  the  component  with  largest 
sample  size  (x^-n^-l  where  n^  *  max  {n^  and  xi=nit  1  S  i  5  k,  i  ^  j). 
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This  method  is  of  special  interest  as  it  was  designed  for  the  practical  but 
difficult  case  of  unequal  sample  sizes.  The  above  methods  are  a  fairly 
complete  sample  of  methods  presently  known  to  work  for  series  systems  of  order 
greater  than  three.  There  is  also  the  weighting  method  [9]  which  is 
applicable  to  series  systems  of  high  reliability  but  this  method  requires 
knowledge  of  the  relative  ordering  of  component  unreliability. 


V.  Improvements  in  the  Likelihood  Ratio  Method 

As  originally  introduced  the  Likelihood  Ratio  method  had  difficulties 

when  the  system  was  so  reliable  or  the  samples  taken  so  small  that  some 

components  are  tested  without  a  resulting  failure.  Since  this  is  often  the 

case  for  expensive  highly  reliable  systems  such  as  missiles,  an  initial 

attempt  was  made  to  correct  this  difficulty  by  adjusting  the  number  of  failures 
til 

to  4/kn^  for  the  i  component  when  there  were  no  actual  failures  out  of  n^ 
tests  for  the  iC^  component.  Thus  this  factor  will  be  smaller  as  the  order 
of  the  system,  k,  and/or  as  n^  increases.  The  adjusted  binomial  data  is  then 
used  by  the  likelihood  ratio  method  resulting  in  a  method  referred  to  as  LRL. 

When  testing  the  components  it  is  known  that  no  component  is  perfect  but 
time  and  expense  constraints  limit  the  test  size  of  the  different  components  so 
that  for  highly  reliable  systems,  some,  many  or  all  of  the  components  may  not 
register  a  failure.  On  the  other  hand  if  a  component  in  a  highly  reliable 
system  registers  one  failure  out  of  fifteen  trials,  the  lack  of  failures  on 
the  other  components  may  suggest  that  the  component  is  more  than  .933  reliable. 
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Thus  it  seems  desirable  to  adjust  the  failures  (and  successes)  used  by  the 

likelihood  ratio  method  to  reflect  the  overall  reliability  of  the  components. 

A  method  that  will  do  such  an  adjustment  for  normally  distributed  data,  by 

shrinking  everything  towards  the  grand  mean,  is  Stein's  estimator  [5], 

In  [6]  it  is  shown  how  to  adapt  this  technique  to  binomial  data:  Let  x^, 

1  £  i  £  k  ,  be  the  number  of  successes  out  of  n.,  1  £  i  £  k  trials;  let  y^= 

n^arcsin^x^/n^)  +  1),  where  y^  ~  NCo^.l).  Stein's  emperical  Bayesian  method 

~  _  k  _  £ 

can  be  applied  so  that  y  =  y  +  (l-(k-3)/s)  (y  -  y)  where  s  =  Z  (y  -  y)  and 

k  *  1  1  1  1 

y  =  (  Z  y  )/k;  then  r  =  .5(l+sin  y./Vn.)  is  the  new  estimate  of  success 

t  i  i 

Jl  /v  ^ 

rate  and  x.  =  r.n.  is  the  Stein's  Estimate  of  successes.  This  method  was 
i  li 

tried  but  the  results  were  often  contradictory  with  failure  rates  going  away 
from  the  "average"  rather  than  shrinking  towards  it.  In  [3],  [4]  and  [6] 

"limited  translations  estimators"  were  defined  and  explained.  They  were  appealing 
since  they  do  not  allow  the  estimates  to  shift  "too  far"  from  the  maximum 

*  A 

likelihood  estimate.  Using  the  above  notation:  y.=  y  +(l-{(k-3)/v}<J>(u.))(y.-y)  , 

k  1  1  1 

K  2  -  2 

where  v  =  max  {n,  Z  (y.-y)  }  and  u.  =  (  (y.-y)  k(k-3)  )/((k-l)v)  and 
1  1  1  1 

<t>(0  =  min{l,  Dg^Ck-l) /Vu^}  where  Dg0  is  the  value  given  in  [4].  After  this 
"limited  translation  Steins  estimate"  of  the  binomial  data  is  complete  it  used 
by  the  likelihood  ratio  method  to  get  an  estimate,  referred  to  as  LRS  13. 

Preliminary  testing  of  this  method  indicated  that  it  might  have  some  very 
desirable  features  even  being  a  <$=.01  or  <$=.02  bound  it  was  noticeably  closer 
to  the  true  reliability  than  other  methods. 

Since  the  component  test  data  is  binomially  distributed  and  not  normally 
distributed  an  empirical  Bayesian  approach  was  directly  applied  to  the  binomial 
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f 


data.  Let  X^jp  ~  B(n^,p)  where  p  is  distributed  as  Beta  with  p.d.f. 


/  \  r  (n)  r-1  ,,  Nn-r-l 

8(P)  =  r (r) r (n-r)  p  (1"p) 


0  <  p  <  1 


where  n  >  r  >  0.  The  marginal  distribution  of  is  used  to  obtain  point 


estimates  for  r  and  n,  ?  and  n.  The  posterior  distribution  of  p|x.  has 

r+x .  ,  1 

i  i  i  R 

We  estimate  E[p  x.]  by  -sr- -  ,  ~  . 

ri  l  J  n+n.  ,  where  r  =  L  x./k  and 

1  i=i  1 

k 

£  x. 


r+x, 

E[p lx. ]  = 

1  i  n+n . 


n  = 


i=l 


k 

E  Xi 
(  -=■  ) 
i=l  ^  n.  ’ 

l 


Note  that  it  follows  that  if  n^=N  for  all  i  then  n=N  and  r  is  the  average  of 
the  successes.  The  resulting  adjusted  empirical  Bayesian  binomial  test  data 
is  used  in  the  Likelihood  Ratio  Method.  The  resulting  confidence  bounds 
are  referred  to  as  LRB  9.  Both  LRS  13  and  LRB  9  methods  had  to  be  adjusted 
for  the  case  when  no  failures  occurred  for  any  of  the  components.  For  these 
methods  the  zero  failure  case  is  computed  by  adding  .01  to  the  confidence 
bound  obtained  when  only  one  failure  occurs,  a  failure  on  the  component  with 
the  largest  sample  size. 


I 


1 


VI .  Simulation  Model  and  Methods  of  Comparison 

As  all  the  methods  presented,  both  old  and  new,  are  approximate  there  is 
no  way  to  analytically  show  superiority.  To  establish  advantages  or 
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disadvantages  of  the  methods  relative  to  each  other  they  must  be  compared 
on  a  large  number  of  systems  with  variety  in  number  of  components,  component 
sample  sizes  and  system  reliability. 

To  do  these  comparisons  the  methods  were  applied  to  simulated  data  from  a 
series  system  of  order  k,  with  component  i  being  tested  n^  times,  1  J  i  <  k. 

Each  system  was  simulated  100  times  since  a  larger  number  of  simulations  resulted 
in  the  same  type  of  results.  The  use  of  100  simulations  is  checked  periodically 
by  simulating  200  to  500  times  and  checking  these  results  with  the  100 
simulation  case.  For  each  of  the  100  simulations  for  a  method  we  counted  the 
number  of  times  the  computed  lower  confidence  bounds  were  below  the  true 
reliability,  below  the  true  reliability  plus  .01,  and  below  the  true  reliability 
plus  .02;  also  the  standard  deviation  of  the  estimates  about  the  true 
reliability  was  computed  to  give  a  measure  of  closeness. 

The  simulations  were  run  for  80%,  90%  and  95%  lower  confidence  bounds  but 
in  the  interest  of  brevity  the  results  will  be  summarized  only  for  the  important 
case  of  90%  bounds.  The  simulations  were  run  for  systems  with  k=6,  10,  15,  20 
and  30  components,  with  true  system  reliability  R  =  .85,  .868,  .886,  .90,  .93, 

.96  and  with  the  following  component  sample  sizes:  equal  n^  with  n^=  15,  20, 

30  or  40;  unequal  n^  with  n^  taking  on  the  values:  (5,  10,  20,  30),  (15,  30, 
60),  (20,  30,  40,  50),  (20,  20,  20,  20,  100)  or  (5,  20,  20,  20,  100)  where 
these  sample  sizes  are  distributed  proportionally  according  to  the  order,  k, 
of  the  system.  This  covers  many  of  the  cases  that  arise  in  practice  and 
gives  enough  data  to  see  the  trends. 

To  summarize  the  results  for  a  set  of  cases  it  is  necessary  to  compare 
e.'_  h  method  to  the  others  and  to  try  to  determine  an  overall  "best"  method. 
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In  the  tables  that  follow  method  A  is  said  to  have  beaten  method  B  if 

1.  Both  are  exact  90%  bounds  (6=.01,  90%  bounds,  S=.02,  90%  bounds,  or  neither 
result  is  a  bound)  and  the  estimated  standard  deviation  of  A  is 
smaller  than  that  of  B. 

2.  Method  A  is  a  5=.01,  90%  bound  and  Method  B  is  an  exact  90%  bound  but 
the  estimated  standard  deviation  of  A  is  less  than  the  estimated 
standard  deviation  of  B  minus  .005. 

3.  Similarly  if  A  is  a  o=.02,  90%  bound  and  B  is  a  6=.01,  90%  bound 

but  the  estimated  standard  deviation  of  A  is  less  than  the  estimated 

standard  deviation  of  B  minus  .005. 

4.  A  is  a  6=.02,  90%  confidence  bound  and  B  is  an  exact  90%  bound  but 

the  estimated  standard  deviation  of  A  is  less  than  that  of  B  minus  .01. 

5.  Method  A  cannot  beat  Method  B  if  A  is  neither  an  exact  90%,  a  6=.01 
or  <$=.02,  90%  bound  and  B  is  at  least  a  6=.02,  90%  bound. 

Using  this  criterion  the  following  tables  show  how  each  method  did  relative 
to  the  others  and  which  methods  seem  best  overall  for  the  set  of  cases  considered. 


I 
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k=.  6/  Equal  Sample  Sizes  ,  n  .  =  15  or  20/9  Sets  of  Simulations 


exact  bound 


overall 


ot  a  bound 


AO 

MML I 

MLG3 

LRL 

LRS13 

LR39 


Pairwise  Comparison  of  Methods  for  IjComparison  of  Individual! 
the  Sets  of  Simulations  (!  Methods  with  all  Others 


AO  MML" 


9  i 

8  ; 

8 

5 

3 

8  ■ 

8 
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•2 
•1 

i  LP.S  1 3  fe-1 
1 


cuel  Samole  S 


30  cr  Z4Q  ,8  Se 


AO 
MML  I 
MLG3 
LRL 
-.RSI  3 


Pairwise  Comparison  of  Methods  for  Comparison  of  Indivicua 
the  Sets  of  Simulations  [Methods  with  all  Others 
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AO  !  -  8-0 
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MLC-3  1 3-5  8-0 
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LP.S  13 
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5-3  1  7-1 

0-8  i  0-8 


8-0  6-2 

7- 1  7- 

8- 0  7- 


7-1  i  7-1 
7-1  :  6-2 


1-7  i  4-4  „ 

.-7  I  0-8  |r 

■-7  !  1-7  || 

.-7  :  2-6  ! 

i  7-1  )i 


3  1/2  -1  1/2 
Q-5 

1- 4 

2- 3 
5-0 

3  1/2-  1  1/2 
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k=  6,  14  Sets  of  Simulations,  Unequal  Sample  Sizes 
=  20,20,20,20,20,100  or  n.=  5,20,20,20,20,100 


Pairwise  Conoarison  of  Methods  for  iComoarison  of  Individual! 


the  Sets  of  Simulations 


‘Methods  with  all  Others 


i  AO  1 

•MML  I 

£-1 
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6-8 

i 

•  LRL 
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i LRS13  P-l- 

I LRB9 
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k=  6  /  9  Sets  of  Simulations,  Ur.ecrual  Samole  Sizes 

=  15,30,60,15,30,60  or  n,=  20,3*0,^0,50,20,30, 


oest 

overall 


AO 

MML  I 
MLGB 
LRL 
LRS13 
\Ju33. 


TABLE  hi 
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k=  6  ,  9  Sets  of  Simulations,  Unequal  Sample  Sizes 

ni=  T5, 15, 15, 15, 15 ,75  or  ni=  5,15,15,15,15,75 


best 

overall 

a  -exact  bound 
\ =0  (exact)  o  =  .01  -=.02 

i 

not  a  bound  ! 

AO 

1 

9  !  i 
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4  i 

5 
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2  i  2 

5 
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1 
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1 
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2 
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k= 6  ,  5  Sets  of  Simulations,  Uneoual  Samole  Sizes 

ni=  5,10,20,30,5,10 
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TABLE  jy 

k=io,  Equal  Sample  Sizes  ,  n.  =  i5  or20  , lySets  Simulations 


TA3LE  VI 
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k=15,  Equal  Sample  Sizes  ,  n.=15  cr20  , lOSets  of  Simulations 
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TABLE  VIII 

k= 20,  Equal  Sample  Sizes  ,  n-=  15  or  20  ,  7  Sets  of  Simulations 
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the  Sets  of  Simulations  iMethods  with  all  Others 
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k=  20,  10  Sets  of  Simulations,  Oreruai  Sar.ole  Sizes 
n.=  5, 10 ,20, 30 ,5,10,20, 30, 5, 10 ,20, 30 ,5,10,2b,  30, 5, 10, 20, 30 
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VII.  Conclusions 

From  the  summary  data  in  the  above  tables  we  can  see  some  interesting 
patterns  and  develop  some  guidelines  for  which  methods  should  be  used  and  for 
what  types  of  systems.  In  particular  it  is  clear  that  for  many  standard, 
"textbook"  situations,  the  AO  method  is  superior;  that  is,  the  tables  show 
that  if  there  are  a  large  number  of  components  in  a  series  system  and  if 
all  the  components  have  equal,  moderate  to  large,  sample  sizes,  then  the 
approximate  properties  of  AO  take  hold  and  it  gives  the  "best"  bounds.  On  the 
other  hand  if  the  situation  is  complicated  by  small  component  sizes  or  if 
the  component  sample  sizes  are  unequal,  then  the  AO  method  is  worse  than  many 
of  the  other  methods  due  to  its  being  too  conservative  and  having  the  largest 
estimated  standard  deviation  about  the  true  reliability  (in  all  the  simulations 
AO  was  at  least  a  6=.02,  90%  bound  and  in  most  cases  it  was  a  6=0  bound). 

In  the  series  situations  that  most  closely  resemble  those  found  in 
actual  practice,  i.e.  either  the  sample  sizes  are  small  or  the  sample  sizes 
are  unequal  the  methods  based  on  the  Likelihood  Ratio  method  are  consistently 
better  than  all  the  others.  In  particular  for  6  or  a  larger  number  of  components 
with  equal  but  small  sample  sizes  (n  =  15  or  20),  LRS  13  is  consistently 
the  best  though  often  it  is  a  90%  bound  with  6=.01  or  .02  rather  than  6=0. 

For  large  sample  sizes,  LRS  13  is  too  optimistic  and  is  often  not  even  a 
6=.02,  90%  bound. 

A  more  consistent  method  in  the  equal  sample  size  case  is  LRB  9  which  is 
second  to  AO  in  the  equal,  not  small,  component  sample  size  case  and  second  to 
LRS  13  in  the  equal,  small  component  sample  size  case.  LRB  9  seldom  fails 
to  be  a  bound  losing  to  LRS  13  only  because  it  is  not  as  close  to  the  true 
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reliability  as  LRS  13.  When  LRB  9  loses  to  AO  it  just  barely  loses. 

In  the  most  complicated,  bur  also  most  realistic,  case  where  the 
sample  size  of  the  components  of  the  system  vary  the  LRB  9  and  LRS  13 
methods  are  consistently  superior  to  the  other  methods.  This  is  partially 
due  to  the  fact  that  the  Likelihood  Ratio  is  not  as  conservative  in  the 
unequal  sample  case  as  are  the  other  methods.  Again  LRS  13  sometimes 
fails  to  be  even  a  6“.02  bound,  whereas  LRB  9  is  consistently  a  bound  and 
in  these  cases  it  is  as  close  or  closer  to  the  true  reliability  than 
LRS  13 .  For  the  unequal  sample  case  the  only  other  method  that  comes 
close  to  either  LRB  9  or  LRS  13  is  MLGB  which  does  well  only  in  one  type 
of  case:  small  unequal  sample  sizes  (n^  varies  amont  5,  10,  20,  30). 

MMLI  and  AO  do  not  do  well  at  all  for  the  unequal  sample  cases  for  they  are 
so  conservative  that  they  are  usually  100%  bounds  with  high  estimated  standard 
deviations. 

For  computing  (90%)  lower  confidence  bounds  for  series  systems  from 
binomial  test  data  we  can  therefore  recommend:  a.  Use  the  AO  method  for  the 
equal  sample  size  with  moderate  to  large  sample  size  (n=40  or  more), 
b.  Use  LRS  13  for  equal  small  sample  size  case.  c.  Use  LRB  9  in  all 
cases  since  it  is  very  comparable  to  the  AO  in  the  AO's  best  cases  and  is 
often  better  than  LRS  13  in  the  unequal  sample  size  case. 


Page  Twenty-Three 


References 


[1]  Borsting,  J.R.  and  Woods,  W.M.  "A  Method  for  Computing  Lower  Confidence 
Limits  on  System  Reliability  using  Component  Failure  Data  with  Unequal 
Sample  Sizes"  1968  -  U.S.  Naval  Postgraduate  School. 

[2]  Easterling,  R.G.  "Approximate  Confidence  Limits  for  System  Reliability" 

JASA  67  (1972)  p.  220-222. 

[3]  Efron,  B.  and  Morris,  C.  "Limiting  the  Risk  of  Bayes  Estimations  - 
Part  1:  The  Bayes  Case"  JASA,  66  (1971)  p.  807-815. 

[4]  _ "Limiting  the  Risk  -  Part  2:  The  Empirical  Bayes  Case" 

JASA  67  (1972)  p.  130-139. 

[5]  _ "Steins  Estimation  Rule  and  its  Competitors,  An  Empirical 

Approach"  JASA  68  (1973)  p.  117-130. 

[6]  _  "Data  Analysis  Using  Stein's  Estimator"  JASA  70  (1975) 

p.  311-319. 

[7]  Madansky,  A.  "Approximate  Confidence  Limits  for  the  Reliability  of  Series 
and  Parallel  Systems"  Technometrics  7  (1965)  p.  495-503. 

[8]  Mann,  N.R.  and  Grubbs,  F.E.  "Approximately  Optimum  Confidence  Bounds  for 
Systems  Reliability  Based  on  Component  Test  Data"  Technometrics  16  No. 3 
(1974)  p.  335-347. 

[9]  Myhre,  J.M.,  Rosenfeld,  A.M. ,  Saunders,  S.C.  "Determining  Confidence 
Bounds  for  Highly  Reliable  Coherent  Systems  Based  on  a  Paucity  of 
Component  Failures,"  Naval  Research  Logistics  Quarterly  25  (1978) 

p.  213-227. 

[10]  Myhre,  J.M.  and  Saunders,  S.C.  "On  Confidence  Limits  for  the  Reliability 
of  Systems,"  Annals  of  Mathematical  Statistics  39  No. 5  (1968)  p.  1463-1472. 

[11]  Myhre,  J.M.  and  Saunders,  S.C.,  "Comparison  of  Two  Methods  of  obtaining 
Approximate  Confidence  Intervals  for  System  Reliability",  Technometrics  10 
No.  1  (February  1968)  p.  37-49 

[12]  Winterbottom,  Alan  "Lower  Confidence  Limits  for  Series  System  Reliability 
from  Binomial  Subsystem  Data"  JASA  69  (1974)  p.  782-788. 


